The Spike Process: a Simple Test Case for Independent or Sparse Component Analysis

We examine curious behaviors of the Independent Component Analysis (ICA) and Sparse Component Analysis (SCA) when they are applied to some simple stochastic processes called the “simple” and “generalized” spike processes. Both processes put a single spike at a random location in the zero vector of length for each realization. The simple spike process puts a unit impulse whereas the generalized spike process puts a sample from the standard normal distribution. We obtained interesting set of theorems for these processes. The behavior of SCA to these processes turned out to be much simpler than that of ICA. Our results are useful for validating any ICA/SCA software package because it is very easy to simulate these processes and the desirable answers are known from our theorems.